An Overview of NTCIR-5 QAC3
نویسندگان
چکیده
This paper provides an overview of NTCIR-5 QAC3 (Question Answering Challenge 3). QAC is a series of challenges for evaluating question answering technologies in Japanese. QAC3 follows the same course as QAC based on the success of the previous two workshops, with its task limited to that corresponding to QAC2 Subtask 3 aiming at the convergence of research resources for novel subjects. This task assumes interactive use of QA systems and evaluates, among other things, the abilities needed under such circumstances, i.e. proper interpretation of questions under a given dialogue context; in other words, context processing abilities such as anaphora resolution and ellipses handling (hereafter we refer to the task as the IAD task, where IAD stands for Information Access Dialogue, and to the whole workshop as QAC3). The IAD task in QAC3 is based on QAC2 Subtask 3 with several improvements, including elaboration of the scope of questions and answers and introduction of multi-grade evaluation and the concept of a correct answer set. In addition, a new WoZ method was devised and applied in the QAC3 test set construction. QAC3 had as many participants as QAC2 Subtask 3, and new trials and advances in existing methods were observed from the
منابع مشابه
Question Answering Experiments at NTCIR-5: Acquisition of Answer Evaluation Patterns and Context Processing using Passage Retrieval
In our participation in QAC3, our Question Answering system developed for QAC2 is extended in two ways, separately. The first system is constructed to improve the performance of answer evaluation. The automatic lexico-syntactic pattern acquisition from large corpora and the method to incorporate the patterns into QA system are developed and evaluated. The second system is constructed to impleme...
متن کاملThree Systems and One Verifier - HOKUM's Participation in QAC3 of NTCIR-5
This paper is a report from collective participation in NTCIR-5 Question Answering Challenge between researchers from Mie University, Hokkaido University and Otaru University of Commerce. Although our results were not impressive, we would like to share our experiences with everyone who think about participating in the challenge but is afraid of his or her lack of experience in the field. Unders...
متن کاملSynthesis of Multiple Answer Evaluation Measures using a Machine Learning Technique for a QA System
The present paper proposes a new method that synthesizes answer evaluation rules using layered neural networks. A Base Question Answering System that employs a combined conventional method (NUT-BASE system) is implemented and evaluated in the NTCIR-5 workshop Question Answering Challenge 3 (QAC3). Based on the evaluation results, the authors focus on performance improvement for the list task an...
متن کاملAnswering Contextual Questions Based on the Cohesion with the Knowledge - Yokohama National University at NTCIR-5 QAC3
In this paper, we introduce the notion “the cohesion with the knowledge”, and, based on it, propose a question answering system to answer contextual questions using a non-contextual QA system. The contextual questions usually have some cohesive relation to their context like reference expressions. Therefore, systems have to detect the cohesion and resolve the reference relations to answer the q...
متن کاملOverview of NTCIR-11
This is an overview of NTCIR-11, the eleventh sesquiannual workshop for the evaluation of Information Access technologies. NTCIR-11 presents the most diverse set of evaluation tasks in the history of NTCIR, led by over 40 cutting-edge researchers worldwide. This paper presents a brief history of NTCIR and overall statistics of NTCIR-11, followed by an introduction of nine evaluation tasks. We c...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2005